Skip to main content
This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal

HCL Notes/Domino 8.5 Forum (includes Notes Traveler)

HCL Notes/Domino 8.5 Forum (includes Notes Traveler)
Previous Next

The problem is about numeric precision

Doubles use 52 bits to store the mantissa, which corresponds to a precision of slightly over 15 decimal digits. Hence the limitation you are observing. I simply missed that point when I quickly responded to your post (which is why I later edited my response to correct my mistake).


Feedback response number JSAK9G9VQY created by ~Chris Frofootherobu on 02/12/2014

modulo of big numbers (~Ben Asakilying... 12.Feb.14)
. . This thread on Stackoverflow looks ... (~Fritz Ekfoober... 12.Feb.14)
. . Bit by bit? (~Martha Lopjipy... 12.Feb.14)
. . Try this [edited] (~Fred Asatumibu... 12.Feb.14)
. . . . has a limit too (~Ben Asakilying... 12.Feb.14)
. . . . . . The problem is about numeric precis... (~Fred Asatumibu... 12.Feb.14)
. . Split it up into chunks (~Mark Froboosim... 13.Feb.14)




Printer-friendly

Search this forum

Member Tools


RSS Feeds

 RSS feedsRSS
All forum posts RSS
All main topics RSS